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A system processes an image containing a first line and a second line, where the first and second lines includes a plurality of pixels, 
to generate an interpolated line. The system selects a first and second set of pixels from the first line and second line, respectively, 
and generates a first set and second set of filtered values by filtering the first set and second set of pixels with a first and second filter, 
respectively. The system identifies in the first line an edge location in the first set of the filtered values by a first filtered value of the first 
set of filtered values being at least one of less than and equal to a first predetermined value and a second filtered value of the first set of 
the filtered values being at least one of greater than and equal to the first predetermined value. The system also identifies in the second 
line an edge location in the second set of filtered values by a first filtered value of the second set of the filtered values being at least one 
of less than and equal to a second predetermined value and a second filtered value of the second set of filtered values being at least one of 
greater than and equal to the second predetermined value. TTie system then interpolates based on the edge location of the first line and the 
edge location of the second line to generate an interpolated pixel of the interpolated line. 
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DESCRIPTION 

SYSTEM FOR CONVERSION OF INTERLACED VIDEO TO 
PROGRESSIVE VIDEO USING EDGE CORRELATION 

BACKGROUND OF THE INVENTION 

The present invention relates to a video format 
conversion system, and in particular to a video format 
conversion system for converting interlaced video into 
progressive video. 

Video displays generally have a plurality of 
scan lines that are displayed at regular intervals. 
Traditional displays, such as those used in regular 
television displays, have used interlaced scan displays. 
Interlaced systems divide the scan lines of each frame of 
video into two fields, each of which consists of one half 
(every other line) of the total number of scan lines in 
the frame. The resolution of interlaced scan displays is 
limited because each field is separately displayed with 
the lines of the alternate field being black. As an 
example of an interlaced system, in an NTSC system each 
interlaced field is displayed every l/60th of a second 
and a complete frame is displayed every 1/3 Oth of a 
second (two fields) . 

Higher quality displays, such as high 
definition television, use a progressive scan display 
system. In a simple progressive scan display system, the 
number of scan lines visible at a given instant is twice 
that which is visible in a corresponding interlaced 
system. Progressive video means that every line of the 
image signal is successively displayed instead of every 
other line as contemplated by the term interlaced video. 
In other words, the term progressive video means not 
interlaced video. 

The conversion from interlaced video to 
progressive video may be accomplished by scanning the 
interlaced video at twice the horizontal rate to double 
the number of scan lines of video information. Each line 
of interlaced video of a field is displayed and the 
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missing lines (that would normally be black) between the 
lines of the interlaced video are interpolated. 

Many systems for interpolating video pixel 
information of interlaced video to progressive video are 
5 known. Such systems can be broadly categorized into one 
of three types. 

The first type of interpolation system is 
intra-field interpolation which involves interpolating 
the pixels of lines not scanned by using the pixels of 

10 scanned lines only in the current field. Normally such 
systems result in a deterioration of image quality 
because each line is interpolated in some manner by the 
mean value of pixels on directly adjacent upper and lower 
lines with respect to the line being interpolated. The 

15 simplest intra-field system averages the value of the 
pixel immediately above and the value of the pixel 
immediately below the interpolated pixel to obtain an 

average pixel value. While filling the black regions 

» 

between interlaced lines of video , this particular intra - 
20 field method neither provides for sufficient image 

quality nor eliminates flicker associated with small 
details • 

The second type of interpolation system is 
inter-field interpolation which involves interpolating 

25 lines not scanned during the current field by using 

scanned lines of the current field together with scanned 
lines of past and/or future fields, without any motion 
compensation. Although this method may result in high 
image quality for video with stationary scenes, severe 

30 artifacts arise for video portions that involve motion. 
An alternative inter-field interpolation method stores 
one field and uses it to fill the spaces in the following 
field. This approach is satisfactory except for video 
portions that involve motion because a moving object will 

35 be at different locations in adjacent fields. In other 
words r such an interpolated frame, consisting of two 
superimposed fields, will present a moving object at one 
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location on even lines and at another location on odd 
lines, thus producing a double image of the object. 

The third type of interpolation system is 
interpolation with motion compensation which attempts to 
5 solve the problems encountered with the presence of 
motion associated with the intra-field interpolation 
method and the inter-field interpolation method without 
motion compensation* However , motion compensation 

« 

normally requires substantially more computation, at 

10 increased time and expense. Many interpolation systems 
try to compensate for the presence of motion by sampling 
pixel values in an area around the desired pixel and 
extending the sampling of pixels to past and future 
fields in the region of the desired interpolated pixel. 

15 Difficulties of motion compensation for interlaced video 
include compensating for the sampling of pixels that are 
not in the same spatial location in the immediately prior 
and immediately subsequent fields of the field including 
the interpolated pixel. Also, in the event of a scene 

20 change, i.e., an event involving a substantial amount of 
video change, such systems tend to fail. 

Lee et al., U.S. Patent No. 5,428,397, disclose 
a video format conversion system for converting 
interlaced video into progressive video using motion- 

25 compensation. The Lee et al. system uses both an intra- 
field technique and an inter-field technique to achieve 
the conversion. The system determines if the current 
pixel is stationary, and if so then an inter-field 
technique is used to determine the value of the 

30 interpolated pixel. In contrast, if the system 

determines that the current pixel is in motion, then an 
intra-field motion compensating technique is used to 
determine the value of the interpolated pixel. In 
particular, the intra-field technique taught by Lee 

35 et al. uses a simple luminance mean calculating circuit 

connected to a pair of line delay circuits. Accordingly, 
the relatively fast inter-field technique is used on the 
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non-motion portions and the relatively slow intra-field 
technique is used on the motion portions for which the 
relatively fast inter-field technique is not suitable. 

Bretl, U.S. Patent No. 5,475,438, discloses a 
5 pixel interpolation system for developing progressive 
line scan video from two interlaced fields of video, 
using both an intra-field technique and an inter-field 
technique. An intra-field pixel value is determined by 
averaging the pixel luminance in the line above and the 

10 pixel luminance in the line below the desired pixel. A 
motion value is determined indicative of motion of the 
image. An inter-field pixel value is determined by 
comparing the intra-field pixel value with the 
corresponding pixels in the previous and subsequent 

15 frames. The motion value is used to proportion the 

intra-field and the inter-field pixel values to compute a 
value for the desired pixel. The system taught by Bretl 
is computationally complex, thus requiring expensive 
electronics and excessive time to compute. 

20 Simonetti et al., in a paper entitled A 

DE INTERLACE!* FOR IQTV RECEIVERS AND MULTIMEDIA 
APPLICATIONS, disclose a system for converting interlaced 
video to progressive scan video. Simonetti et al. 
suggest that their objective is to improve picture 

25 quality with a fast, low-cost device by using simple but 
effective algorithms with an optimized design flow and 
silicon technology suitable for ASIC developments. The 
system includes motion detection by computing the 
absolute luminance differences between selected pixels, 

30 and thereafter selecting either an intra-field or inter- 
field interpolation technique. When motion is detected, 
an intra-field technique is used to attempt to avoid 
smearing the borders of objects in the proximity of the 
pixel. Simonetti at al. attempt to keep the blurring low 

35 by performing an interpolation along a direction of high 
correlation of the luminance data in the vicinity of the 
pixel. The algorithm taught by Simonetti et al. includes 
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performing an extensive number of comparisons 
(conditional executions) for each pixel and then 
selecting the proper action based upon one or more of the 
comparisons. Unfortunately, in conventional RISC 
5 processors conditional executions, which are used to 

perform such comparison operations, require substantial 
time to execute. Accordingly/ to maximize computational 
speed when using a RISC processor, the number of 
conditional executions should be minimized, 

10 Hong, U.S. Patent No. 5,493,338, discloses a 

system for converting interlaced video to progressive 
video by utilizing a three dimensional median filter. 
The three dimensional median filter applies a weighing 
factor to pixel luminance components of horizontal, 

15 vertical, and diagonal directions of the pixel 

component's periphery to be interpolated by horizontal, 
vertical, and time axis judging signals. However, Hong 
does not use edges in the video when determining pixel 
interpolation which may result in decreased image 

20 quality. 

Marsi et al., in a paper entitled VLSI 

IMPLEMENTATION OF A NONLINEAR IMAGE INTERPOLATION FILTER, 

disclose the conversion of interlaced video to 
progressive video by interpolating along either the 

25 vertical or the horizontal direction using an operator 

function. The operator function is designed to preserve 
edges within an image. However, such an operator 
function does not account for image movement or provide 
adequate results for non-horizontally or non-vertically 

30 oriented edges. 

Dong-Il, U.S. Patent No. 5,307,164, discloses a 
system for conversion of interlaced video to progressive 
video by linearly interpolating through a slant 
correlation of a low-pass interpolation signal and 

3 5 through ,f 0" insertion of a high-pass interpolation signal 
in a vertical direction. The slant correlation technique 
involves obtaining the mean luminance value of highly 
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correlated pixels, wherein the correlation of the pixels 
is detected in diagonal directions and vertical 
directions by a series of calculations. Thereafter, the 
greatest correlation (smallest mean value) is determined 
5 by a series of comparisons and used to select which two 
pixels to average in order to calculate the interpolated 
pixel . 

Patti et al., U.S. Patent No. 5,602,654, 
disclose a two-step, contour sensitive deinterlacing 

10 system especially suitable for obtaining still images 
from interlaced video. The first step determines for 
each missing pixel of an interlaced field whether the 
absolute difference between the luminance of the pixels 
above and below the missing pixel is greater than a 

15 preselected threshold value. If it is decided that the 
missing pixel lies at a low-vertical frequency location, 
its value is estimated via vertical interpolation. 
Otherwise, the second step determines whether or not 
there is a well-defined contour passing through the 

20 missing pixel, and determines the contour's direction if 
there is one by comparing blocks of pixels. In the 
presence of a well defined contour, the missing pixel is 
obtained by averaging the intensity values along the 
direction of the contour in the field lines immediately 

25 above and below the missing field line. However, the 

* 

computational requirements necessary to process the 
blocks of data, as taught by Patti et al . , to determine 
the contours are excessive. 

Martinez et al . , in a paper entitled SPATIAL 
3 0 INTERPOLATION OF INTERLACED TELEVISION PICTURES, teach a 

system for converting interlaced video to progressive 
video by an algorithm that attempts to determine edges of 
features within an image by using corresponding sets of 
pixels of two adjacent lines of interlaced video. The 
35 algorithm is based on a line shift model, in which small 
segments of adjacent raster scan lines are assumed to be 
related by a spatially varying horizontal shift. The 
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algorithm taught by Martinez et al. involves two steps. 
The first step is to estimate of the velocity obtained 
from a window of image samples surrounding the point of 
interest. For color pictures, this operation is carried 
5 out only on the luminance component. In order to compute 
the velocity estimate Martinez et al. teach that a set of 
linear equations must be solved for each point, which is 
computationally intensive and potentially requires 
expensive electronics. The second step involves 

10 projecting the velocity estimate onto the two lines 

adjacent to the desired interpolated pixel and the image 
intensities at the corresponding points are averaged 
together. For color pictures, this operation takes place 
on each component color. 

15 Conceptually the algorithm taught by Martinez 

et al. can be thought of as fitting a surface to a series 
of points on two adjacent interlaced lines of video. 
Then at the point of interest the gradient is determined, 
which is the direction of the steepest slope on the 

20 surface to that point of interest. The perpendicular to 
that gradient is the flattest portion at that point on 
the surface and is considered an edge to a feature. The 
interpolation is performed with pixels generally along 
that perpendicular to maintain the contour of the edge. 

25 Accordingly, Martinez et al. teach that pixel values from 
both lines are used to compute the interpolated line of 
pixels. Further, when proceeding to the next set of two 
lines of interlaced video, one line of which was used for 
the interpolation of the preceding interpolated line, 

3 0 there is no reusable data to simplify the calculations 
because of the "surface" nature of the calculations. 

What is desired, therefore, is a system for 
converting interlaced video to progressive video that is 
not computationally intensive and results in retaining 

3 5 edge information for features within the video. 
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SUMMARY OF THE INVENTION 

The present invention overcomes the 
aforementioned drawbacks of the prior art by providing a 
system for processing an image containing at least a 
5 first line and a second line, where each of the first and 
second lines includes a plurality of pixels, to generate 
an interpolated line where the interpolated line includes 
a plurality of interpolated pixels located intermediate 
the first and second lines. The system selects a first 

10 set of pixels from the first line and a second set of 

pixels from the second line and generates a first set of 
filtered values by filtering the first set of pixels with 
a first filter and a second set of filtered values by 
filtering the second set of pixels with a second filter. 

15 The system identifies in the .first line at least one edge 
location in the first set of the filtered values by a 
first filtered value of the first set of filtered values 
being at least one of less than and equal to a first 
predetermined value and a second filtered value of the 

20 first set of the filtered values being at least one of 

greater than and equal to the first predetermined value. 
The system also identifies in the second line at least 
one edge location in the second set of filtered values by 
a first filtered value of the second set of the filtered 

25 values being at least one of less than and equal to a 

second predetermined value and a second filtered value of 
the second set of filtered values being at least one of 
greater than and equal to the second predetermined value. 
The system then interpolates based on the at least one 

30 edge location of the first line and the at least one edge 
location of the second line to generate at least one of 
the interpolated pixels of the interpolated line. 

In an alternative embodiment the system 
includes a third line which includes a plurality of 

35 pixels and the interpolated pixels are located 

intermediate the first and third lines. The system 
selects a third set of pixels from the third line and 
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generates a third set of filtered values by filtering the 
third set of filtered values with a third filter. The 
system selects at least one of a plurality of putative 
edge features based upon a first selection criteria where 
5 each putative edge feature is defined by a set of 

filtered values including at least one filtered value of 
the first set of filtered values, at least one filtered 
value of the second set of filtered values, and at least 
one filtered value of the third set of filtered values. 

10 The first selection criteria is defined by at least two 
of the at least one filtered value of the first set of 
filtered values, the at least one filtered value of the 
second set of filtered values, and the at least one 
filtered value of the third set of filtered values being 

15 at least one of greater than ,and equal to a predetermined 
value and at least one of less than and equal to the 
predetermined value. The system interpolates based on 
selecting at least one of a plurality of putative edge 
features to generate at least one of the interpolated 

20 pixels of the interpolated line. 

In the preferred embodiment a fourth line of 
pixels is used and the putative edge feature includes the 
fourth line of pixels. The system then includes a second 
selection criteria defined by at least two filtered 

25 values of , a filtered value of the first set of filtered 
values, a filtered value of the second set of filtered 
values, and a filtered value of the third set of filtered 
values being the non-selected one of the at least one of 
the greater than and equal to the predetermined value and 

30 the at least one of the less than and equal to the 

predetermined value of the putative selection selected as 
a result of the first selection criteria. The system 
then interpolates based on selecting at least one of a 
plurality of putative edge features including the first 

35 selection criteria and not the second selection criteria 
to generate at least one of the interpolated pixels of 
the interpolated line. 
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In both embodiments the system converts 
interlaced video to progressive video using a technique 
that is not computationally intensive and results in 
retaining edge information for features within the video. 
5 Also, the technique maintains edge details of an image 
when encountering luminance changes, such as shadows 
obscuring an edge feature. The principal technique used 
to maintain such edge features is by not directly using 
the pixel luminance values themselves but instead using 

10 "binary" edge features which provides higher resultant 
image quality. Another advantage of the system is that 
each set of calculations performed on a single line of 
interlaced video may be used for both the interpolated 
pixels above and below the line of interlaced video, 

15 without recalculation of the pixels values using the 
filter. 

The foregoing and other objectives , features, 
and advantages of the invention will be more readily 
understood upon consideration of the following detailed 
20 description of the invention, taken in conjunction with 
the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of an exemplary 
25 system that includes an intra-field interpolator, an 
inter-field interpolator, and a motion detector. 

FIG. 2 is a representation of interlaced video 
and progressive video. 

FIG. 3 is a representation of pixels, a portion 
30 of two lines of interlaced video. 

FIG. 4 is a set of sample pixel luminance 

values. 

FIG. 5 is a set of filtered values obtained by 
filtering the pixels of FIG. 4 with a [-1+2 -1] filter. 
35 FIG. 6 is a set of polarity values of the 

filtered values of FIG. 5 used to identify edge features. 

FIG. 7 is a set of thresholded values 

■ 

containing primarily zeros with scattered positive (+) 
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and negative (-) values obtained by passing a set or 
filtered values through a threshold filter. 

FIGS. 8-12 show a set of exemplary selection 
criteria for the thresholded values of FIG. 7. 
5 FIG. 13 shows a set of vertical edge conditions 

for the selection criteria of FIG. 12. 

FIG. 14 shows an alternative set of vertical 
edge conditions for the selection criteria of FIG. 12. 

FIG. 15 shows a set of vertical edge conditions 
10 for one of the selection criteria of FIG. 8. 

FIG. 16 shows an edge selection test. 

FIG. 17 shows an edge putative selection for FIG. 16. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

15 Referring to FIG. 1, a combination intra-frame 

and inter-frame motion adaptive interpolation system 8 
includes an input luminance signal Y in . Y in is used for 
interpolating a line within the field to be interpolated 
by an intra-field interpolator 10 and an inter-field 

20 interpolator 12. Based upon the judgment of a motion 

detector 14 that judges motion within the input image, a 
pixel to be interpolated within a region corresponding to 
the motion is interpolated either with the intra- frame 
interpolator 10 or with the inter-field interpolator 12. 

25 It is to be understood, that the system of the present 

invention may include any suitable system for inter-field 
determinations and motion detection. Also, the system 
may include only an intra-field interpolator, if desired. 

Referring to FIG. 2, as previously described 

30 interlaced video has an image spread over two different 
fields of a frame. In order to convert interlaced video 
20 to progressive video 30 for displays such as high 
definition television, the interpolated pixels between 
the scan lines of the interlaced video 20 need to be 

35 determined. A technique that is chosen to determine the 
interpolated pixels should provide a high quality image 
free from artifacts. 
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Referring to FIG, 3, an interpolated pixel X 
may be calculated from a given set of sequential pixels 
A-G of a portion of a line of interlaced video and 
sequential pixels H-N of a portion of a subsequent line 
5 of interlaced video. Preferably, pixels H-N are from the 
next line of interlaced video. It is to be understood 
that the system can use any suitable n umb er of pixels 
above and below the interpolated pixel X. Preferably, 
the pixels are both sequential in order and symmetrical 

10 around the interpolated pixel X. 

Many current interpolation techniques encounter 
difficulty maintaining edge details of an image when 
encountering luminance changes, such as shadows obscuring 
an edge feature. The inventor came to the realization 

15 that many current interpolating techniques directly use 
the pixel luminance values of different interlaced lines 
of video to identify edge features within the image. In 
contrast to the identification of edge features by 
directly using the pixel luminance values themselves, the 

20 inventor determined that the correlation of "binary" edge 
features provides higher resultant image quality. 

The inventor observed that fitting a "surface," 
as taught by Martinez et al . , requires the simultaneous 
computation using pixel intensity values from a portion 

25 of both surrounding lines for each interpolated pixel of 
an interpolated line. When computing the interpolated 
pixels for the next interpolated line, the pixel 
intensity values of the shared line of interlaced video 
(the line between the two interpolated lines) are reused 

30 to recompute the next "surface." In contrast to the 

teachings of Martinez et al., the inventor came to the 
realization that using an interpolating technique that 
computes a function, or otherwise uses a filter, for each 
portion of an interlaced line of video, where the 

35 function of each line is independent of the other lines 

of interlaced video, permits a subsequent calculation for 
the next interpolated line to reuse the prior calculation 
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performed on the shared line of interlaced video. In 
other words, each set of calculations performed on a 
single line of interlaced video may be used for both the 
interpolated pixels above and below the line of 
5 interlaced video. Using this insight, the inventor 
postulated that typical contours representative of an 
edge feature along a line of an image may be determined 
from either a second order filter or a high pass filter - 
Alternatively, any other suitable filter, such as for 

10 example, Guassian or Sobel, may be used. Such edge 

features may include intensity changes from low luminance 
to high luminance, or intensity changes from high 
luminance to low luminance. 

Referring to FIG. 4, a set of sample pixel 

15 luminance values are shown. An exemplary second-order 
zero-crossing filter to be applied to the luminance 
values may be [-1 +2 -1]. The [-1 +2 -1] filter 
calculates the second derivative of three adjacent pixels 
of the data. The zero order crossing filter is applied 

20 to each pixel of an interlaced line of pixels by 

calculating the summation of the preceding luminance 
pixel value multiplied by -1, the current pixel value 
multiplied by +2, and the subsequent pixel value 
multiplied by -1. The set of filtered values resulting 

25 after filtering the pixels of FIG. 4 with the [-1 +2 -1] 
filter is shown in FIG. 5. The resulting magnitudes of 
the filtered values from the zero-crossing filter applied 
to a line of interlaced video are not important in 
determining the existence of edge features, but rather 

30 the polarity values (positive or negative) of the 

filtered values from the zero-crossing filter, as shown 
in FIG. 6, are used to identify edge features. Changes 
from positive (+) to negative (-) , or from negative (-) 
to positive (+) of adjacent pixels identifies an edge 

35 feature at that location. In addition, a change from 
positive (+) to negative (-) to positive (+) , or from 
negative (-) to positive (+) to negative (-) of adjacent 
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pixels may either be indicative of an edge feature or 
noise. A change from negative (-) or positive ( + ) to 
zero (0) may indicate an edge feature if desired. 

It is to be understood that the zero-crossing 
5 filter may use non-adjacent pixels, if desired. The 

number of pixels used to calculate each positive, zero, 
or negative filtered value of from a set of luminance 
values may be selected, as desired. In addition, the 
particular filter selected may result in a set of 

10 filtered values that are not centered around zero for the 
identification of edge features. In such a case the 
identification of edge features may be based on a change 
from "less than" or "equal to" a predetermined value to 
"greater than" or "equal to" the predetermined value. 

15 Alternatively, the change may be from greater than or 

equal to the predetermined value to less than or equal to 
the predetermined value. Filtered values that match the 
predetermined value (zero or non-zero) may be used as 
indicative of edge features, if desired. Alternatively, 

20 filtered values that match the predetermined value may be 
disregarded and the filtered values to either side of the 
predetermined value (s) may be used. 

The inventor realized that if a principal 
purpose of the interpolation system is to detect and 

25 maintain edges within the image, then the general 

characteristic change within each set of filtered values 
of two or more interlaced lines indicative of an edge 
feature should correlate if an edge passes through them. 
Accordingly, the inventor came to the realization that 

30 the edge features of two or more lines of interlaced 

video should be related to each other. The edge features 
can be tracked across the gap between the interlaced 
lines by identifying the edge feature closest to the 
column in the line above the line in which pixel X is 

35 located. Referring again to FIG. 6, for the upper line 
(the line above the interpolated pixel X) the closest 
edge feature is column 3 to column 4 as the sign changes 



14 



WO 99/1 4944 PCT/ JP98/04 1 09 

from positive (+) to negative (-) . The change from 
negative (-) to zero (0) of column 4 to column 5 may be 
indicative of an edge feature, albeit likely a smaller 
edge feature than a polarity change. In addition, the 
5 negative (-) to zero (0) to positive (+) of columns 4-6 
may be ignored, if desired. 

Next, the lower line (the line below the 
interpolated pixel X) is searched for an edge feature on 
the opposite side of the interpolated pixel X from the 

10 edge feature identified in the other line, which is 
indicated by the positive (+) to negative (-) edge 
feature of columns 4 to column 5 (FIG. 6) . If multiple 
edge features (transitions) are located in the 
neighborhood of X in the lower line, then the transition 

15 location for which the lower .line pixel luminance most 
closely matches the pixel luminance of the edge feature 
in the upper line is selected as the matching point. 
There is a limited region of columns about the 
interpolated pixel X over which edge features are 

20 searched for, such as, for example, three columns to 
either side of the interpolated pixel X. After iden- 
tifying the edge transitions about the interpolated 
pixel X, a linear interpolation is performed along the 
direction defined by the edge. Referring again to 

25 FIG. 6, the linear interpolation of the interpolated 

pixel X is the pixel luminance of pixel 50 and pixel 52, 
namely, (9+10)/2. Alternatively, any other suitable 
interpolation technique may be used based on the edge 
features • 

30 It is to be understood that the calculations 

performed on the upper and the lower lines may be 
reversed, if desired. Further, the lines of interlaced 
video used to interpolate pixel X do not necessarily need 
to be the lines immediately above and below the 

35 interpolated pixel X. 

As previously described, the aforementioned 
technique is described in terms of either a positive, a 
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negative, or a zero value. Alternatively, depending on 
the zero-crossing filter selected, which may include an 
offset factor, the comparisons for identifying edge 
features may be values other than positive and negative. 
5 In the event that no edge correlation is 

determined, a vertical interpolation technique is used. 

An alternative interpolating technique uses a 
high pass filter on the luminance values, such as [+1 +3 
-3 -1]. The [+1 +3 -3 -1] high pass filter identifies 

10 the high frequency content of a set of pixels which is 
indicative of an edge feature. The high pass filter is 
applied to each pixel of an interlaced line of pixels by 
calculating the summation of the second preceding 
luminance pixel value multiplied by +1, the preceding 

15 pixel value multiplied by +3,. the current pixel value 

multiplied by -3, and the next pixel value multiplied by 
-1. Alternatively, the high pass filter may be shifted 
to the right so the preceding pixel value is multiplied 
by +1, the current pixel is multiplied by +3, the next 

20 pixel is multiplied by +1, and the next pixel value is 

multiplied by +3. The filtered values from the high pass 
filter of a line of interlaced video is then applied to a 
threshold function to assign filtered values the value of 
zero that have an absolute value less than a 

25 predetermined threshold. This sets lower frequency 

components to the same value, preferably zero, as less 
indicative of edge features. The remaining non-zero 
values indicate the existence of high frequency features 
within the image, such as an edge feature. The 

3 0 magnitudes of the resulting non-zero filtered values from 

♦ 

the high frequency filter applied to a line of interlaced 
video are not important in determining the existence of 
edge features but rather the polarity values (positive or 
negative) from the thresholded high pass filter are used 
35 to identify edge features. Changes from positive (+) to 
negative (-) , or from negative (-) to positive (+) of 
adjacent pixels identifies an edge feature at that 
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location • In addition, a change from positive (+) to 
negative (-) to positive (+) of adjacent pixels or from 
negative (-) to positive (+) to negative (-) of adjacent 
pixels may either be indicative of an edge feature or 
5 noise. 

It is to be understood that the high pass 
filter may use non-adjacent pixels if desired. The 
number of pixels used to determine each positive , zero, 
or negative filtered value may be selected as desired, 

10 In addition, the particular high pass filter used may 

result in a set of filtered values that are not centered 
around zero to which a threshold filter is applied. In 
such a case the threshold filter would eliminate, or 
otherwise set to a constant, a range of filtered values 

15 indicative of low frequency features of the image. In 
such a case the identification of edge locations may be 
based on a change from "less than 1 ' or "equal to" a 
predetermined value to "greater than" or "equal to" the 
predetermined value. Alternatively, the identification 

20 of edge locations may be based on a change from greater 
than or equal to the predetermined value to less than or 
equal to the predetermined value. Preferably, the 
predetermined value would be the value to which the low 
frequency features are set, such as zero. 

25 With the proper selection of the threshold 

values the resulting matrix of values contains primarily 
zeros with scattered positive (+) and negative (-) 
values, such as exemplified by FIG. 7. The preferred 
technique includes the use of two lines above and two 

30 lines below the interpolated pixel X. The system 

compares the polarity values of the filtered pixel values 
on the lines above and the lines below the interpolated 
pixel X among a set of different general directions about 
the interpolated pixel X. The system may use any 

35 suitable set of filtered pixel values about the 

interpolated pixel X. By extending the system to include 
two lines above and two lines below the interpolated 
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pixel X the system can more readily identify significant 
edge features. If desired, the system may use fewer or 
additional lines above and/or below the interpolated 
pixel X. 

5 Based on the general angle being examined about 

pixel X, or otherwise the selected set of filtered 
values, the preferred putative selection criteria are the 
same positive or negative sign values in the line 
directly above and the line directly below the interpo- 

10 lated pixel X and at least one of the second line above 
and the second line below the interpolated pixel X. The 
multiple sets of pixels at different angles may be 
selected as desired, such as those shown in FIGS. 8-12. 
In the examples of FIGS. 8-12, selections 60 and 62 match 

15 such a criteria. Alternatively, the criteria may be any 
suitable number of the same sign value on any set of 
suitable lines in a general direction, or otherwise any 
selected set of filtered values, about the interpolated 
pixel. 

20 For those sets of pixels that match the 

putative selection criteria then a second test is 
preferably performed. For each initial match the system 
checks the filtered values to ensure that an opposing 
edge of opposite sign does not cross the putative edge 

25 feature. The opposing edge may be checked in the 

vertical direction across the line at intermediate pixel 
locations from each end of the line (set of pixels) . 
Referring to FIG. 13, for the putative selection 62 the 
seven vertical edge conditions 64a-64g would be checked, 

30 as shown, to ensure that negative (alternate polarity) 

values are not present on both sides of the selection 62. 
Alternatively, referring to FIG. 14 , the system may check 
vertical edge conditions 66a-66g of pixels above and 
below the selection 62, such as shown in FIG. 14. 

35 Referring to FIG. 15, the putative selection 60 has a 

greater slope than the putative selection 62 of FIG. 12. 
Accordingly, several fewer checks are necessary to 
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validate the non-crossing of an opposite polarity edge. 
Accordingly, for generally horizontal lines the greater 
number vertical opposing edge conditions to be checked 
increases the likelihood of removing that putative line 
5 as a candidate. This counterbalances the false tendency 
of such low angle horizontal lines to indicate an edge 
feature because the filtered values used for the edge 
feature are distant from one another , and may in fact be 
related to different edge features. The more vertically 

10 oriented lines are less likely to be removed by such an 
additional opposing edge condition check. 

It is to be understood that the vertical 
conditions may check pixel values close to or more 
distant from the line as desired, and need not actually 

15 cross the line of the putatiye edge feature (e.g., the 
opposing check pixels are all to one side of the line) . 
In addition, the vertical conditions may be at an oblique 
angle, if desired. Preferably , the vertical pixels 
checked only extend the length of the pixels indicating 

20 the putative edge feature • 

Alternatively, for those sets of pixels that 
match the putative selection criteria an alternative 
second test may be performed. For each initial match the 
system checks the filtered values to ensure that an 

25 opposing edge of opposite sign does not cross the 

putative edge feature. The opposing edge may be checked 
in both an oblique angle and a vertical direction by 
checking groups of pixels on either side of the putative 
edge feature. Referring to FIG. 16, a set of three upper 

30 pixels 80a-80c are checked to determine if any has a 

negative (alternate polarity) value and a set of three 
lower pixels 82a-82c are checked to determine if any has 
a negative (alternate polarity) value. If a negative 
(alternative polarity) value is determined for at least 

3 5 one of the upper pixels 80a-80c and at least one of the 
lower pixels 82a-82c then the putative selection 62 is 
removed as an edge feature candidate. Also, a set of two 
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upper pixels 84a-84b and a set of two lower pixels 
86a-96b are checked in the same manner as pixels 80a-80c 
and 82a-82c. In addition, a set of three upper pixels 
88a-88c and a set of three lower pixels 90a-90c are 
5 checked in the same manner as pixels 80a-80c and 82a-82c. 
Using blocks of pixels allows for the detection of 
opposing edges at different oblique angles with fewer 
comparisons than would be required if each combination of 
the pixels in the respective blocks were compared to each 
10 other. 

It is to be understood that the pixel groups 
may check pixel values close to or more distant from the 
line, as desired, and need not actually cross the line of 
the putative edge feature (e.g., the opposing check 

15 pixels are all to one side of the line) . Also, the pixel 
groups may extend only the length of the pixels 
indicating the putative edge feature, if desired. The 
pixel groups may include any suitable number of pixels 
and number of separate groups, as desired. 

20 It was determined by the inventor that 

frequently the putative edge that satisfies either of the 
aforementioned second tests may still not be an actual 
edge feature. In many cases such erroneous edge features 
may be determined by mere coincidence in regions of an 

25 image that contains excessive texture. To determine if 

« 

the putative edge is more likely an actual edge feature 
the system may further include checking for same polarity 
edges that cross the putative edge feature. The system 
preferably does not consider pixels close to the putative 

30 edge, such as within one pixel distance from the line, to 
permit edge features of greater width to still be 
considered edge features. The preferred selection of the 
same polarity pixels are those within a rectangle defined 
by the limits of the length of the line and greater than 

35 one pixel distant from the line. If any of the preferred 
selection of the same polarity pixels have the same 
polarity then the putative edge feature is not selected. 
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Referring to FIG. 17 , for the putative 
selection 62 the preferred selection of the same polarity 
pixels would be 96a-96e and 98a-98e. Alternative pixels 
and the criteria used for matching the same polarity 
5 pixels may be selected, as desired. 

In the event of no edge correlation, a vertical 
interpolation technique is used. 

Alternatively, a match among any three or four 
pixels values of the four lines of interlaced video along 
10 in a general direction about the interpolated pixel X may 
be used. In addition, additional lines of interlaced 
video may be used, if desired. 

The edge feature identification and selection 
criteria described in relation to the zero-crossing 
15 filter and the high pass filter may be switched with one 
another, if desired. In addition, any of the previously 
described techniques are likewise suitable for use with 
curved edges and arcs. 

The terms and expressions which have been 
20 employed in the foregoing specification are used therein 
as terms of description and not of limitation, and there 
is no intention, in the use of such terms and expres- 
sions, of excluding equivalents of the features shown and 
described or portions thereof, it being recognized that 
25 the scope of the invention is defined and limited only by 
the claims which follow. 
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CLAIMS 

1. A method of processing an image containing 
at least a first line and a second line, where each of 
said first and second lines includes a plurality of 
5 pixels, to generate an interpolated line where said 

interpolated line includes a plurality of interpolated 
pixels located intermediate said first and second lines, 
said method comprising the steps of: 

(a) selecting a first set of said pixels from 
10 said first line; 

(b) generating a first set of filtered values 
by filtering said first set of said pixels 
with a first filter; 

(c) identifying at least one edge location in 
15 said first set of said filtered values by 

a first filtered value of said first set 
of said filtered values being at least one 
of less than and equal to a first 
predetermined value and a second filtered 
20 value of said first set of said filtered 

values being at least one of greater than 
and equal to said first predetermined 
value; 

(d) selecting a second set of said pixels from 
25 said second line; 

(e) generating a second set of filtered values 
by filtering said second set of said 
pixels with a second filter; 

(f) identifying at least one edge location in 
30 said second set of said filtered values by 

a first filtered value of said second set 
of said filtered values being at least one 
of less than and equal to a second prede- 
termined value and a second filtered value 
35 of said second set of said filtered values 

being at least one of greater than and 
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equal to said second predetermined value; 
and 

(g) interpolating based on said at least one 

said edge location of step (c) and said at 
5 least one said edge location of step (f) 

to generate at least one of said 
interpolated pixels of said interpolated 
line. 



10 2. The method of claim 1 wherein said first 

set of said pixels includes at least three pixels and 
said second set of said pixels includes at least three 
pixels. 



15 3. The method of .claim 1 wherein at least one 

of said first filter and said second filter is a high 
pass filter. 

4. The method of claim 1 wherein at least one 

20 of said first filter and said second filter is a second 
order filter. 



5. The method of claim 1 wherein said first 
line and said second line are adjacent lines of one field 

25 of an interlaced video image. 

6. The method of claim 1 wherein at least one 
of said first predetermined value and said second 
predetermined value is zero. 

30 

7. The method of claim 1 wherein said first 
filter and said second filter are independent of each 
other. 



35 8. The method of claim 4 wherein said at 

least one of said first filter and said second filter is 
[-1 2 -1]. 
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9. The method of claim 3 wherein said at 
least one of said first filter and said second filter is 
[+1 +3 -3 -1] . 

5 10. The method of claim 1, further comprising 

the steps of: 

(a) selecting a third set of pixels from said 
third line; 

(b) generating a third set of filtered values 
10 by filtering said third set of said pixels 

with a third filter; 

(c) identifying at least one edge location in 
said third set of said filtered values by 
a first filtered value of said third set 

15 of said filtered values being at least one 

of less than and equal to a third 
predetermined value and a second filtered 
value of said third set of said filtered 
values at least one of being greater than 

20 and equal to said third predetermined 

value ; and 

(f) interpolating based on said at least one 

said edge location of said second line and 
said at least one said edge location of 
25 said third line to generate at least one 

interpolated pixel of an interpolated line 
located intermediate said second and third 
lines. 

30 11. The method of claim 10 wherein said second 

set of said filtered values is only calculated once. 

12. The method of claim 1 wherein said first, 
second, and third predetermined values are the same 
35 value. 
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13 • A method of processing an image containing 
at least a first line, a second line, and a third line, 
where each of said first, second, and third lines 
includes a plurality of pixels, to generate an 
5 interpolated line where said interpolated line includes a 
plurality of interpolated pixels located intermediate 
said first and third lines, said method comprising the 
steps of: 

(a) selecting a first set of said pixels from 
10 said first line, a second set of said 

pixels from said second line, and a third 
set of pixels from said third line; 

(b) generating a first set of filtered values 
by filtering said first set of said pixels 

15 with a first filter, a second set of 

filtered values by filtering said second 
set of said pixels with a second filter, 
and a third set of filtered values by 
filtering said third set of said pixels 

20 with a third filter; 

(c) selecting at least one of a plurality of 
putative edge features based upon a first 
selection criteria where each putative 
edge feature is defined by a set of 

25 filtered values including at least one 

filtered value of said first set of 
filtered values, at least one filtered 
value of said second set of filtered 
values , and at least one filtered value of 

30 said third set of filtered values, and 



by at least two of said at least one 
filtered value of said first set of 
filtered values, said at least one 
35 filtered value of said second set of 

filtered values, and said at least one 
filtered value of said third set of 
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filtered values being at least one of 
greater than and equal to a predetermined 
value and at least one of less than and 
equal to said predetermined value; and 
5 (d) interpolating based on said selecting at 

least one of a plurality of putative edge 
features to generate at least one of said 
interpolated pixels of said interpolated 
line. 

10 

14. The method of claim 13 wherein said first 
set of said pixels, said second set of said pixels, and 
said third set of said pixels includes at least three 
pixels* 

15 

15. The method of claim 13 wherein at least 
one of said first filter, said second filter, and said 
third filter is a high pass filter. 

20 16. The method of claim 13 wherein at least 

one of said first filter, said second filter, and said 
third filter is a second order filter. 
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17. The method of claim 13 wherein said first 
line, said second line, and said third line are adjacent 
lines of one field of an interlaced video image. 

18. The method of claim 13 wherein said 
predetermined value is zero. 



19. The method of claim 13 wherein said 
predetermined value consists of a first predetermined 
value, a second predetermined value , a third 
predetermined value each of which is used to define said 
35 first selection criteria with respect to said first line, 
said second line, and said third line, respectively. 
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20. The method of claim 19 wherein said first, 
second, and third predetermined values are the same 
value . 

5 21. The method of claim 13 wherein said first 

filter, second filter, and said third filter and said 
second filter are independent of each other. 

22. The method of claim 13 wherein at least 
10 one of said first filter, said second filter, and said 

third filter is [-1 2 -1]. 

23. The method of claim 13 wherein at least 
one of said first filter, said second filter and said 

15 third filter is [+1 +3 -3 -1). 

24. The method of claim 13 wherein said first 
set of said filtered values, said second set of said 
filtered values, and said third set of said filtered 

20 values are only calculated once. 

25. The method of claim 13, further comprising 
the steps of: 

(a) selecting a fourth set of said pixels from 

25 a fourth line; 

■ 

(b) generating a fourth set of filtered values 
by filtering said fourth set of said 
pixels with a fourth filter; 

(c) selecting at least one of a plurality of 
30 said putative edge features based upon 

said first selection criteria where each 
putative edge feature further includes 
said at least one filtered value of said 
fourth set of filtered values and said 
first selection criteria is defined by at 
least three of said at least one filtered 
value of said first set of filtered 
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values, said at least: one filtered value 
of said second set of filtered values, 
said at least one filtered value of said 
third set of filtered values, and said at 
5 least one filtered value of said fourth 

set of filtered values being at least one 
of greater than and equal to a 
predetermined value and at least one of 
less than and equal to said predetermined 

10 value; and 

(d) interpolating based on said selecting at 
least one of a plurality of putative edge 
features to generate at least one of said 
interpolated pixels of said interpolated 

15 line. 

26. The method of claim 13 further comprising 
the steps of: 

(a) said selecting at least one of a plurality 

20 of putative edge features is based upon 

said first selection criteria and said 
second selection criteria, said second 
selection criteria is defined by at least 
two filtered values of a filtered value of 

25 said first set of filtered values, a 

filtered value of said second set of 
filtered values, and a filtered value of 
said third set of filtered values being 
the other one of said at least one of said 

30 greater than and equal to said 

predetermined value and said at least one 
of said less than and equal to said 
predetermined value selected for said 
first selection criteria; and 

35 (b) interpolating based on said selecting at 

least one of a plurality of putative edge 
features including said first selection 
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criteria ana not said second selection 
criteria to generate at least one of said 
interpolated pixels of said interpolated 
line. 

5 

27. A system for processing an image 
comprising: 

(a) an intra-field interpolator receives at 
least a portion of a first line of said 

10 image and a portion of a second line of 

said image, where each of said first and 
second lines includes a plurality of 
pixels each of which has a corresponding 
pixel luminance value; 

15 (b) said intra-field interpolator in response 

to receiving said portion of said first 
line and said portion of said second line 
filters said corresponding pixel luminance 
values to generate a first set of filtered 

20 values for said first line and a second 

set of filtered values for said second 
line where each of said filtered values of 
said first set and each of said filtered 
values of said second set are 

25 characterized as at least one of a first 

* 

condition and a second condition where 
said first condition is characterized as 
being at least one of greater than and 
equal to a predetermined value and said 
30 second condition is characterized as being 

at least one of less than and equal to 
said predetermined value; and 
(c) said intra-field interpoltor generates an 
interpolated pixel located intermediate 
said first and second lines based on said 



29 



PCT/JP98/04109 

first condition and said second condition 
of said filtered values of said first and 
second lines. 
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